Multi-Layered Image-Based Rendering

In this article, we describe a multi-layered image-based rendering system that can use different types of input to produce and render new environments. Each separate input that can be manipulated independently is called a layer. In our implementation, the types of layers that can be manipulated are the image-based and 3-D-based layers. The computation required for rendering the newly-crafted environment is reduced by using cached composite snapshots of that environment at reference poses. These cached snapshots are used to directly generate novel views, and the original layers are used only when necessary. Another key concept is the identification of types of holes generated as a result of pixel transfer from the composite snapshots. For optimal rendering quality, the algorithm used in filling these holes is specific to the hole type (either intralayer or interlayer). The ideas embodied in our multi-layer IBR system are useful in augmenting the capabilities of applications that require fast and geometrically consistent rendering of 3-D scenes such as video editing.